🕷️️ Job Radar • SCRAPING

freelancer.com 🟢 2026-05-09

🔹 Extract complete dataset from filter-based list on a KSA/GCC-accessible website
👤 Client: 🇸🇦 Al Hufuf, Saudi Arabia Member since 2019-12-21
💰 Price: $27 Average bid
🚩 Problem: Automate data extraction from a website with pagination and filters.
📦 Existing: Not specified

Specifications:

[Target] Extract all available data points from each item in the list, including metadata such as date, location, and description.
[Method] Implement an automated scraping solution using Python or similar language to handle dynamic content loading and pagination.
[UI/UX] Ensure the scraper can navigate through filters and load additional pages without user interaction.
[Stack] Use libraries like BeautifulSoup, Selenium, or Scrapy for web scraping. Consider headless browsers if necessary.
[Security] Comply with website terms of service and legal requirements. Implement rate limiting to avoid detection.
[Format] Output data in a structured format such as JSON or CSV.

Workflow:

Analyze the website structure, identify elements that need interaction (e.g., filters, pagination buttons).
Set up a development environment with necessary tools and libraries for web scraping.
Write scripts to automate navigation through filters and page loading. Handle dynamic content using JavaScript execution if needed.
Test the scraper on a small subset of data to ensure accuracy and efficiency.
Run the full scraper, monitor its performance, and adjust as necessary to handle any issues or changes in website structure.